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LIST MODELS OF PROCEDURE LEARNING 


Michael P. Matessa and Peter G. Poison* 


Ames Research Center 


1. INTRODUCTION 


This paper presents a new theory of the initial stages of skill acquisition and then employs the theory 
to model current and future training programs for flight management systems (FMSs) in modem 
commercial airliners like the Boeing 777 and the Airbus A320. The theoretical foundations for the 
theory are a new synthesis of the literature on human memory and the latest version of the ACT-R 
theory of skill acquisition (Anderson, Bothell, Byme, Douglass, Lebiere, and Qin, in press; 

Anderson and Schunn, 2000; Taatgen and Lee, 2003). 

The models derived from the theory describe training programs where to-be-leamed procedures are 
formally trained, and trainees must demonstrate mastery before they can achieve journeyman status 
or go on to more advanced on-the-job training. Airline transition training programs are examples of 
this paradigm. The models generate estimates of the training time required to master the procedures 
specified in the training program syllabus. These estimates and the underlying theory are used to 
evaluate current and future training programs and support construction of business cases justifying 
investments in training-program modifications. 

1.1 Problems and Solutions 

Numerous studies have documented operational and training problems with the modern autoflight 
systems, in particular the FMS and its pilot interface, the control display unit (CDU). During the last 
few years, more attention has been given to the limitations of current autoflight training methods. 
Many studies have concluded that current training programs are inadequate in both depth and 
breadth of coverage of FMS functions (Air Transport Association, 1997, 1998, 1999; BASI, 1998; 
FAA Human Factors Team, 1996). 

This paper argues that the inadequacies of the programs are due to airline training practices that 
encourage pilots to master FMS programming tasks by memorizing lists of actions, one list for each 
task. Treating FMS programming skills as lists of actions can interfere with acquisition of robust and 
flexible skills. 


* University of Colorado at Boulder, Boulder, CO 80309 
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Blackmon and Poison (2002) argue that a solution to these training problems is to develop new 
training programs based on an intelligent computer-aided instruction technology, cognitive tutors 
(Anderson, Corbett, Koedinger, and Pelletier, 1995; Corbett, Koedinger, and Anderson, 1997). 
Blackmon and Poison claim that cognitive tutors can dramatically increase the depth and breadth of 
autoflight skills while maintaining the same training footprint. However, airlines will not even 
consider radical changes to current training practices until the research community provides sound 
business cases justifying large capital investments and radical modifications to the pilot training 
culture. 

A critical step in building a defensible business case is estimating the time required for both current 
and proposed training programs. This paper presents a new method for estimating training time 
based on a theory of the initial states of skill acquisition that is analogous to the Card, Moran, and 
Newell (1983) Keystroke Level Model for estimating skilled performance by characterizing 
procedures as sequences of actions and drastically simplifying descriptions of the underlying 
cognitive processes. The new method is referred to as the Lists of Actions Model. 

1.2 Outline of The Paper 

The paper first gives the theoretical foundations for the new Lists of Actions Model and describes it 
in detail. Then the model is applied to the learning of FMS procedures. The model makes predictions 
about current training times that agree with times found in empirical research. The model is then 
used to analyze modifications to current training programs. 


2. THEORETICAL FOUNDATIONS 


The foundations of the theory are the Fitts (1964) three-stage model of skill acquisition that 
underlies almost all model analyses of skill acquisition and the latest ACT-R simulation models of 
skill-acquisition processes proposed by Anderson, et al. (2004) and Taatgen and Lee (2003). There 
is widespread agreement that the acquisition of a complex skill involves three stages, first described 
by Fitts (1964). The following description of these stages has been adapted from VanLehn (1996) to 
describe acquisition of the skills necessary to operate an autoflight system. 

2.1 The Three Stages of Skill Acquisition 

The Cognitive Stage involves learning enough about the interface of a system to be able to generate 
descriptions of the steps in a procedure that can be successfully memorized. Initial training on such 
systems includes learning descriptions for elements of displays and controls. The Associative Stage 
involves memorizing a description of the actions required to perform a procedure. This stage ends 
with the successful storage in memory of representations that can be used to reliably generate the 
actions necessary to perform the procedure. The third or Autonomous Stage involves refining, 
strengthening, and transforming the mental representation of the skill to improve both speed and 
accuracy of performance. The endpoint of the process is a mental representation of the procedure 
that supports rapid and accurate performance even in high-workload situations — automaticity. 
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2.2 The Representation of Skills at the End of Formal Training 

The declarative representation of a procedure enables a newly trained operator to perform correctly 
but at some cost. The procedure is executed interpretively by retrieving description, decoding it, and 
generating the actions described by the retrieved information. These processes are error-prone, they 
take extra time, and they impose additional workload. In addition, declarative representations are 
easily forgotten. They can be maintained only by practicing the procedure at least every two or three 
weeks. 

The ACT-R cognitive architecture assumes that all these limitations can be overcome by acquiring a 
procedural representation of the procedure that enables very rapid retrieval of the actions necessary 
to perform it. The procedural representation is generated, refined, and strengthened during 
successful execution of the procedure mediated by the declarative representation. 

However, recent results (Anderson, et al., 2004) support the claim that achieving automaticity can 
take more than 100 repetitions of a procedure. The transformation from a declarative to a procedural 
representation of a skill is ACT-R’s explanation of the processes that describe Fitts’s Autonomous 
Stage. Thus trainees complete their formal training while they are still in the Associative Stage for 
all but the most frequently executed procedures. 

It follows that training time is in large part determined by the number of repetitions of a procedure 
necessary to acquire a useful declarative representation that is resistant to forgetting, i.e., supports 
accurate recall after retention intervals of as long as two weeks. Most of the additional experience 
necessary to convert the declarative representation into the much more durable and flexible 
procedural representation occurs during a trainee’s initial on-the-job experience. However, 
declarative representations enable a trainee to pass a qualifying exam and perform acceptably during 
on-the-job training. 

2.3 Implications of the Human Memory Literature for Understanding the Associative Stage 

Research on human memory during the last 100 years has focused on the learning and retention of a 
wide variety of declarative representations from serial lists of nonsense syllables (Ebbinghaus, 
1888/1913) to expository and native texts (Kintsch, 1998). The theory presented in this paper 
assumes that it is possible to model a training program by analogy to one or more experimental 
paradigms that have been employed to study human memory, e.g., serial or paired-associates list 
learning. The three steps of constructing such a model by analogy are described as follows. 

First, specify an experimental paradigm in the human-memory literature that has a structure 
analogous to that of the training program. Next, identify the features of tasks and training procedures 
that are analogous to variables that have been investigated in the context of the analogous 
experimental paradigm, e.g., mapping list length onto complexity of FMS procedures. Third, review 
the literature on the selected experimental paradigm to derive quantitative relationships between 
measures of learning and retention and the variables that are mapped onto important features of tasks 
and training methods, e.g., list length, distribution of practice, retention interval, and properties of 
list elements. 
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The list-learning literature contains extensive parametric studies of time and trials to mastery and 
retention for these and other variables. These quantitative relationships can be used to derive a 
model of a specific training program and make quantitative predictions of training time. It is also 
possible to propose change in existing training programs that are analogous to experimental 
manipulations in the memory literature that enhance both learning and retention, improving the 
efficiency of a training program. 

2.4 The Loci of Expertise Effects in Skill Acquisition 

The theory developed in this paper hypothesizes that the Cognitive and Associative Stages of skill 
acquisition are the loci of expertise effects in skill acquisition. The characteristics of the learning 
processes for these two stages are determined by the strategies employed by a trainee. The learning 
strategies are determined by interactions between 1) the properties of the system user interface, 

2) the trainee’s expertise acquired from experience with similar systems, and 3) the assumptions 
made by a trainee about how to learn and perform tasks using the system. These interactions also 
determine the experimental paradigm that best models the cognitive and associative stages of skill 
acquisition and the difficulties of these stages. 

The Cognitive Stage involves learning how to encode descriptions that when retrieved from memory 
enable them to generate actions necessary to perform tasks. The Associative Stage involves 
memorizing lists of these descriptions that enable trainees to perform tasks that will be on a 
proficiency examination at the completion of a training program. 

Catrambone (1995) has shown that novices in a domain — in the absence of explicit guidance on how 
to learn skills — assume that skill acquisition involves memorizing lists of actions required to 
perform tasks and that performing a task involves retrieving and performing each memorized action. 
It will be shown that for a novice pilot, learning a new FMS procedure is analogous to learning a 
serial list of nonsense syllables. 

Experts in a domain have much more powerful encoding strategies and are able to impose 
meaningful structures on representations that are well integrated with their domain knowledge. 

Pilots with extensive experience with several different FMS-equipped aircraft show all the attributes 
of domain experts acquiring new knowledge in their domain. For example, they can successfully 
perform a moderately complex procedure after a single demonstration. In addition, experts’ lists tend 
to be much shorter. Experts can generate descriptions that concisely describe a sequence of several 
actions. Finally, experts have specialized cognitive skills that enable them to generate actions when 
retrieval fails during execution of an infrequently performed task. 

2.5 Current Airline Training Practices 

Flight manuals, FMS pilots’ guides, computer-based training (CBT) lessons, and other documents 
all contain very different descriptions of the steps in procedures, ranging from bare lists of key 
presses in flight manuals to detailed descriptions of each function invoked by a key press in a pilots’ 
guide. A demonstration of a procedure in a CBT lesson or by an instructor pilot in a fixed-base 
simulator presents the series of required actions and resulting display changes that occur during 
successful execution of a procedure. 
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None of these methods for training FMS procedures provides pilots with much — if any — clear 
guidance on how to generate memorable — or even meaningful — descriptions of a FMS procedure. 
Pilots using a glass cockpit for the first time tend to focus on actions required to perform a procedure 
because they will be tested on their ability to reproduce the action sequence to demonstrate mastery 
of the procedure during later simulator sessions or on a check ride. 

Learning FMS procedures involves both mastering a specific set of FMS programming tasks and 
acquiring specialized problem-solving skills. The ability to problem solve to use the FMS in an 
unfamiliar situation is based on knowledge of the CDU interface conventions, a retrieval structure 
that segments an action sequence into subsequences along with the knowledge of the purpose of 
each subsequence, and the skills necessary to use this knowledge and cues provided by the interface. 

None of these skills and knowledge is provided to pilots in today’s training programs. Pilots must 
induce them by making generalizations across procedures using the FMS. This induction problem is 
very difficult in the absence of explicit instructions. This is why clear evidence of reconstruction 
skills is seen only in pilots with experience with several different FMS-equipped aircraft. 

In addition, Fennell (2004) 1 observed that pilots have strong default assumptions about learning 
cockpit procedures. Typical procedures are sequences of actions such as button presses, switch 
positionings, and observations of instrument readings and indicators. Memorizing lists is not an 
unreasonable approach to learning required short standard operating procedures (SOPs). However, it 
is a very poor model to apply to the acquisition of FMS programming skills. 

Instructors’ descriptions of their experiences with such trainees are certainly consistent with the 
hypothesis that these pilots try to learn FMS procedures as serial lists of actions. They require 
several repetitions of a procedure before they can perform it without errors. Trainees rapidly forgot 
newly learned procedures, and they must be retrained after a one- or two-day retention interval. 
Furthermore, these trainees’ knowledge of these painfully acquired procedures is very brittle. They 
have trouble generalizing their skills to new procedures that are closely related to a trained 
procedure. 


3. THE LISTS OF ACTIONS MODEL 


A representation of a task must encode both item (actions and parameters) and order information. 
Anderson, Bothell, Lebiere, and Matessa (1998) assumed that item and order information is encoded 
in a hierarchical retrieval structure incorporated in their ACT-R model of serial list learning shown 
in figure 1. The order information is encoded in a hierarchically organized collection of chunks. The 
terminal nodes of this retrieval structure represent the item information. 


' Personal communication, K. Fennell, United Airlines, Oct. 1, 2004. 
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3.1 The Underlying Representation 

The model assumes that pilots transitioning to their first FMS-equipped aircraft master a cockpit 
procedure by memorizing a serial list of declarative representations of individual actions or 
summaries of subsequences of actions. Learning this list takes place during briefings and 
demonstrations of the procedure, study of the procedure as described in a flight manual or other 
training materials, and attempts to execute the procedure as part of a CBT lesson or in a simulator 
session. It is assumed that each of these attempts to learn the list is analogous to a test-study trial in a 
serial recall experiment. 

An interpretive process uses the list to perform the procedure. This process incorporates the 
knowledge necessary to understand each step description and to execute actions necessary to 
perform each step. Thus, an item such as “Press the LEGS key” would generate the actions required 
to locate the Legs key on the CDU keyboard and press it. A parameter such as a waypoint identifier 
would be represented in working memory as a sequence of letters. The interpretative process would 
generate the keystrokes necessary to enter the identifier into the scratch pad. 



Figure 1. List Model Representation. The retrieval structure for a list of nine digits of 

Anderson et al. (1998). 2 

The list actions representation is a consequence of pilots’ decisions to treat the task of mastering 
FMS procedures as learning serial lists of actions. The retrieval structure shown in figure 1 is 
generated by processes that adults use to memorize any arbitrary serial list of items. It is assumed 
that a novice representation of a FMS procedure with nine actions would be represented by replacing 
the terminal-node chunks with chunks representing individual actions in the procedure. 

The retrieval structure only encodes order information and supports access to the chunks 
representing individual actions. The groupings of the actions imposed by this structure have no 


2 Figure 1 is figure 2 copied from Anderson, et al. (1998, p. 348). 
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relationship to the underlying task structure. Because these retrieval structures are unique to each 
task, they block transfer of training. 

3.2 An Example List 

The following is a possible list describing an FMS procedure for the Boeing 777 for responding to 
the following hold clearance that would be generated by a pilot with limited glass-cockpit 
experience. 

NASA 1: Hold west of Haden on the 270° radial. Right turns. 10 mile legs. 

Expect further clearance at 2130. 

The list shown in figure 2 has undesirable properties and would be difficult to memorize. It is 
long — 14 items — and it is organized as a linear sequence of actions that cannot be directly stored in 
memory (Anderson, et al., 1998). Some kind of idiosyncratic organization would have to be imposed 
on it to break it up into sublists before it could be successfully memorized. Furthermore, the 
representation of the procedure for programming a hold shown in figure 2 is specific to a particular 
clearance. It would be relatively easy to generalize this representation to clearances with identical 
parameters but with different values. However, generalizing this procedure to cover the entry of any 
hold clearance requires numerous nontrivial inferences. 


1. Press HOLD Function/Mode Key. 

2. Press LSK 6L, if a holding pattern is already in the route. 

3. Line select waypoint identifier for Haden to scratchpad. 

4. Press LKS 6L. 

5. Enter the quadrant and the radial into the scratchpad, W/270. 

6. Press LSK 2L. 

7. Enter the turn direction into the scratchpad, R. 

8. Press LSK 3L. 

9. Enter the leg distance into the scratchpad, 10. 

10. Press LSK 5L. 

11. Enter expect further clearance time into the scratchpad, 2130. 

12. Press LSK 3R. 

13. Verify the resulting holding pattern on the ND. 

14. Press EXECUTE. 


Figure 2. A possible novice representation of a FMS procedure for responding to a hold clearance. 

3.3 Using the Serial List Learning Literature to Make Training Time Predictions 

The Lists of Actions Model assumes that learning a FMS procedure is analogous to memorizing 
serial lists of nonsense syllables for a pilot with limited FMS experience. Training times can be 
estimated using results of an experimental paradigm initially developed by Ebbinghaus (1888/1913, 
Chapter 8). 
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3.3.1 The Savings Paradigm 

On the first day of the experiment, participants learn a serial list of items to a criterion of mastery of 
one perfect recitation of the list. Items are presented in a fixed order. During the presentation of an 
item in a list, participants must study it and attempt to recall the next item to be presented. 
Participants return to the laboratory 24 hours later and relearn the list to the same criterion of 
mastery. Training stops on the first day that participants perform perfectly on the first presentation of 
the list after a 24-hour retention interval. 

3.3.2 The Relationship Between FMS Procedure Complexity (List Length) and Training 
Performance 

Table 1 presents the number of retentions on each successive day and the number of days of training 
required to be able recall a list perfectly after 24 hours for lists of 4, 8, 12, 16, and 24 items. The 
numbers in the table were derived by synthesizing the results of several experiments from the list- 
learning literature starting with the data from Ebbinghaus (1885/1913, Chapter 8). The numbers are 
extrapolations generated by fitting power functions to Ebbinghaus’ s results and then adjusting them 
to account for the fact that he used a very rapid presentation rate. 

Table 1. Average number of repetitions of a list required to reach a criterion of one 
perfect recall as a function of list length and days. 


List 

Length 

Day 1 

Day 2 

Day 3 

Day 4 

Day 5 

Day 6 

Day 7 

Day 8 

Total 

4 

1.5 

1.2 

1.1 

1 





4.7 

8 

4.5 

2.4 

1.8 

1.5 

1.4 

1 



12.5 

12 

8.5 

3.9 

2.7 

2.1 

1.8 

1.6 

1 


21.7 

16 

15.5 

6.6 

4.2 

3.2 

2.6 

2.2 

2 

1.8 

38.2 

24 

23 

9.5 

5.9 

4.3 

3.4 

2.9 

2 

1.8 

52.8 


Table 1 makes clear that procedures described by long lists with 12 or more items are very difficult 
to master. The 16- and 24-item lists do not reach criterion even after 8 days of training. Furthermore, 
examination of the column for Day 2 shows that much is forgotten after initial mastery of lists on 
Day 1. Underwood (1957) showed that interference caused the large amounts of forgetting shown in 
table 1 and in the original results of the studies from which it was derived. Participants in these early 
studies of list learning participated in many different experiments. The forgetting is caused by 
interference between the representations of a current list and previously learned lists. 

These interference effects are also present in FMS training. A transition training program is 
described by the List Model as involving concurrent mastery of from 20 to 30 lists describing 
various FMS procedures. Many days of practice are required to overcome the effects of interference 
between the memory representations of various procedures. 
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Recall that pilot instructors find that even moderately complex procedures are difficult to train and 
are rapidly forgotten. The impact of list length shown in table 1 and the very powerful interference 
effects found in this list-learning literature account for these problems. 

3.4 Modeling a Training Program 

This section illustrates the methods involved in using table 1 to construct a simplified quantitative 
model of a training program. In this training program, procedures for performing FMS programming 
tasks are introduced during CBT lessons and training sessions with nonpilot instructors. Then, these 
skills are further trained and practiced in briefings and simulator sessions conducted by pilot 
instructors. All actual practice of these skills takes place in fixed-base and full-motion simulators. 

3.4.1 Estimating the Time Required to Train a FMS Procedure 

The model assumes that the training of a specific FMS procedure is analogous to the serial-list- 
leaming experiment described in Section 3.3.1. During any given simulator session, different 
examples of a procedure (e.g., a direction to clearance with different waypoints) are performed 
repeatedly until a pilot can execute the procedure without errors or hesitations. The number of 
required repetitions for a given simulator session and the number of days necessary to master the 
procedure are taken from table 1. 

Training time is estimated by calculating the amount of time it would take to administer N 
repetitions of a procedure of length L during one session in a fixed-base or full-motion simulator. 
The model’s description of the training processes has three time parameters: session setup time 
(SST), repetition setup time (RST), and step time (ST). SST is the time required to set up a simulator 
to begin training a specific procedure. RST is the time required to set up the simulator for the next 
repetition, and ST is the time required for a trainee to perform a step and receive feedback from the 
instructor if necessary. These values are then summed over days to generate a training- time 
prediction for a given procedure. 

The time devoted to training a procedure on one day = SST + N*RST + N*L*ST. 

The values for N, the number of repetitions on a day, are taken from table 1. Values for SST and 
RST were set to 120 seconds, and ST was set to 5 seconds. Current fixed-based and full-motion 
simulators were found to be ill-suited to this kind of training; they are designed to simulate the 
execution of complete missions. 

Figure 3 shows the total time necessary to train a procedure plotted as a function of length, L. The 
function turns out to be linear with a slope of approximately 1 1 minutes per step. 
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Figure 4 shows the percentage of the total training time that was devoted to actually practicing a 
procedure and receiving feedback from an instructor. The figures plotted are the actual training 
times (N*L*ST) summed across days divided by the total training times plotted in figure 2. 



Figure 4. Percentage of training time devoted to actually practicing the procedure. 
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Figure 4 makes very clear the inefficiencies involved in using a mission-oriented simulator to train 
FMS procedures. 

3.4.2 Approximate and More Realistic Models of a Training Program 

A complete FMS training program is modeled by identifying the FMS procedures specified in the 
syllabus and describing each procedure as a list of items to be memorized. The complexity of a FMS 
procedure is represented in the model as the length of a serial list of items describing how to perform 
it. The lists are generated from examples of FMS procedure descriptions in airline and manufacture 
training and reference documentation. An example is shown in figure 1. Pressing a page/mode key 
(e.g., HOLD) or line select key is described as a single list item. Entries into the scratch pad, which 
may involve several keystrokes, are each described as single items. It is similar to descriptions of 
FMS procedures in an airborne flight manual for the Boeing 777. 

An approximate model of a training program can be generated by computing the distribution of list 
lengths and then computing the total time required to train the complete collection of tasks. The 
function fitted to the data in figure 2 can be used to estimate training time for list lengths not 
included in table 1 . 

A more realistic model would be based on a description of an actual Boeing 777 FMS training 
program of a major airline, including a realistic syllabus that specifies the tasks introduced and/or 
practiced on any given day. This model would specify the tasks that are introduced or practiced on 
any day during a multiday training program during a reasonable length simulator session, e.g., four 
hours. 

3.4.3 Using the Approximate Model to Analyze Training Program Modifications 

The following examples assume that a FMS training program can be described as a collection of 30 
lists. Based on a very preliminary study of an actual training program, a hypothetical distribution of 
list lengths was generated and is shown in table 2. In addition, no attempt was made to model a 
realistic syllabus specifying what tasks were introduced or practiced on a given day of the training 
program. 


Table 2. Frequency distribution of list lengths for a hypothetical 
FMS training program. 


List Length 

Frequency 

4 

4 

8 

4 

12 

10 

16 

10 

24 

2 
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3.4.3.1 A model of a current airline training program 

Using the list-length distribution of table 2, a model of a current training program was constructed 
using the same values, 120 seconds, for SST and RST used to compute the predictions shown in 
figures 2 and 3. The total training time was approximately 48 hours. Poison, Irving, and Irving 
(1994) reported that pilot instructors for the Boeing 737-300 fleet of a major airline estimated that 
pilots spent between 15 and 50 hours practicing FMS skills in a simulator. 

This large value for total training time for FMS procedures — 48 hours — is cause by two main 
factors. First, table 1 shows that successful training of complex FMS procedures — lists of 12 or 
more items — requires a very large number of repetitions distributed over at least 8 days. Second, 
figure 3 shows that a full mission simulator is a very inefficient — as well as very expensive — device 
for administering the necessary repetitions. 

3.4.3.2 Introducing part-task trainers 

Numerous studies have shown that PC-based, part-task simulators can be used successfully to train 
skills such as performing FMS procedures (e.g., Salas, Bowers, and Prince, 1998; Salas, Bowers, 
and Rhodenizer, 1998; and Poison, Irving, and Irving, 1994). The lesson planners incorporated into 
commercially developed simulators can be programmed to deliver the necessary repetitions while 
minimizing the SST and RST (Aerosim Technologies, www.aerosim.com; Tricorn Technologies, 
www.tricom-tech.com/products.htm; CAE, www.Cae.com; and Wicat, www.wicat.com). Use of 
such a trainer was modeled by reducing the values of SST or RST to 5 seconds. 

Total training time was reduced to 20 hours. This demonstration indicates that these trainers can 
very efficiently deliver the large number of repetitions necessary to master complex FMS 
procedures, leading to a large reduction in total training time — 20 versus 48 hours. 

3.4.3.3 Training pilots to generate shorter lists 

Pilots do not receive an explicit instruction on how to encode FMS procedures in memory early in 
training. Furthermore, descriptions of procedures in airline training and reference materials are 
inconsistent. The example in figure 2 is consistent with some of the more detailed descriptions of 
FMS procedures in the Boeing 777 Flight Manual of a major airline. Catrambone (1995) has shown 
that novices tend to describe problem solutions in terms of actions used to solve the problem. In the 
case of FMS programming skills, this process leads to long lists that are very difficult to memorize. 

Many studies in memory literature show that subjects can be easily trained to use more effective 
encoding strategies to enhance learning and retention of materials to be memorized. This model 
explores the consequences to teaching pilots methods for generating more concise descriptions of 
FMS procedures, leading to short lists. 

This example assumes that a hypothetical encoding strategy would have little or no effect on 
description list lengths for 4- and 8-item lists, that 12-item lists would be reduced to 8-item lists, that 
16-item lists would be reduced to 12-item lists, and that 24-item lists would be reduced to 16-item 
lists. This encoding scheme would change the distribution of list lengths shown in table 2 to the 
distribution presented in table 3. The column labeled Novice List Length is from table 2. The 
column labeled Trained List Length shows the results of the encoding strategy described previously. 
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Table 3. Frequency distribution of list lengths for a hypothetical FMS training program 
for two different method of encoding lists. 


Novice 
List Length 

Trained 
List Length 

Frequency 

4 

4 

4 

8 

8 

4 

12 

8 

10 

16 

12 

10 

24 

16 

2 


The result of training these encoding skills is to reduce total training time from 48 hours to 29 hours, 
assuming the use of fixed-base and full-motion simulators. If we combine the use of part-task 
trainers with use of more efficient encoding strategies, total training time is reduced to 8.1 hours. 

The encoding and decoding skills assumed in this example are not trivial, and there is no way at this 
point of estimating the amount of time required to train such skills or if they can be taught to pilots 
during the initial stages of transition training to their first FMS-equipped aircraft. However, these 
skills — if mastered early in training — would be useful during later stages of training and line 
operations in supporting more effective acquisition of new skills on the line. 

3.5 Summary of Applications of the Lists of Actions Model 

The examples presented in the preceding sections are illustrations of using results of the serial-list- 
leaming literature to model existing airline training programs and suggest enhancements. Table 4 
summarizes these results. 

Mastering FMS procedures takes a large number of repetitions, especially for procedures longer than 
12 steps (see table 1.) A training program should be set up to deliver the necessary repetitions as 
efficiently as possible by employing part-task trainers. Additional large gains in training efficiency 
can be achieved by teaching pilots to generate shorter descriptions of FMS procedures. 

Table 4. Summary of four modeling experiments. 


Approximate Model of FMS Training 

Total Training Time for 
30 FMS Tasks (hours) 

All Practice in a Full-Mission Simulator 

48 

All Practice Using a Part-Task Trainer 

20 

Efficient Encoding Skills, Practice in Full- 
Mission Simulator 

29 

Efficient Encoding Skills, Practice with 
Part-Task Trainer 

8.1 
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The serial-list-leaming literature describes numerous robust phenomena that have important 
implications for airline training. These phenomena can guide the development of incremental 
changes to existing training programs that can improve pilot performance and reduce training time. 
The examples presented in the preceding section, introducing part-task trainers and more efficient 
encoding strategies, are good illustrations. 

Many training programs introduce FMS procedures early in a three- to six- week transition training 
course. Focus of the program then switches to emergency procedures and other topics. Trainees can 
have very limited or no opportunity to practice flight plan modifications and other FMS procedures 
during the remainder of the program. Results shown in table 1 indicate that forgetting new learned 
procedures is a major problem. Rapid forgetting can be overcome only by distributing opportunities 
for practicing FMS procedures throughout a training program. 

Airline training and reference documentation do not contain complete sets of detailed and 
consistently described FMS procedure descriptions. Thus, pilots new to FMS-equipped aircraft must 
invent their own encoding methods. The list-learning literature shows that providing subjects with 
efficient encoding strategies can facilitate learning. Providing pilots with consistent descriptions of 
procedures and strategies for memorizing them should reduce training time and improve pilot 
performance. 

However, this literature also contains an important lesson. There is no “silver bullet” (Anderson and 
Schunn, 2000). Mastering a complex FMS procedure will require a large number of repetitions that 
must be distributed over many days and then practiced occasionally throughout the rest of a training 
program. Developers of training programs must recognize this fact and then deliver the necessary 
repetitions as efficiently as possible. 


4. SUMMARY 


This paper hypothesizes that initial mastery of an FMS or any other procedure is analogous to 
memorizing a serial list of items. Starting in 1888 with the publication of Ebbinghaus’s monograph 
on memory to the present, a huge collection of empirical and theoretical results has been published 
describing the learning and forgetting of such lists. This literature provides us with a rich collection 
of results that have enabled us to identify problems with current airline training practices and to 
propose many different solutions to these problems. 

The goals of this research program are to validate these insights and develop useful solutions to 
known problems with existing training programs. 
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